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Army  Materiel  Command  and  the  University  of  Alabama  in  Huntsville  part¬ 
nered  to  develop  an  integrated  end-to-end  performance  metrics  system.  The 
integration  includes  data  pulls  from  multiple  data  systems  into  a  metrics 
calculation  and  aggregation  system  that  generates  strategic  performance 
metrics  such  as  Customer  Wait  Time  (CWT),  with  capabilities  spanning  from 
bottom-up  supply  chain  performance  aggregation  capabilities  to  in-depth 
traceability  to  source  (tactical)  level  data  and  documents.  To  support  the 
best  national  defense,  we  must  ensure  that  our  warfighters  receive  the 
supply  support  they  need  in  a  timely  and  efficient  manner.  Supporting  this 
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effort  requires  a  near-real-time  system  that  measures  and  reports  on  supply 
chain  strategic  performance  characteristics  such  as  C WT.  Data  integrity  is 
an  integral  part  of  the  process,  as  is  reaching  common  agreement  on  appro¬ 
priate  data  sources,  algorithms  to  calculate  metrics,  and  the  design  of  a  visual 
dashboard  that  supports  leadership  decisions  and  performance  evaluation, 
with  drill-down  capability  for  lower  level  decision  making. 
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The  current  environment  for  the  DoD  is  one  of  changing 
demands:  recent  high  demand  for  parts  needed  during 
engagement  in  theater  has  been  replaced  with  inventory 
rationalization  due  to  drawdowns,  new  budgetary  pres-  Z 

sures  to  reduce  costs  and  expenditures,  and  a  call  to 
emulate  commercial  practices  of  efficient  supply 
chain  management.  In  order  to  achieve  greater 
management  visibility  across  their  supply  chains. 

Army  Materiel  Command  (AMC),  headquar¬ 
tered  in  Huntsville,  Alabama,  partnered  with 
University  of  Alabama-Huntsville  (UAH)  in  a 
multiyear  project  to  develop  a  supply  chain  per¬ 
formance  metrics  framework  that  provides  a 
more  comprehensive  and  integrated  end-to-end 
supply  chain  metrics  system.  This  research  proj¬ 
ect,  named  the  Enterprise  Supply  Chain  Analysis 
&  Logistics  Engine  (eSCALE)  project,  was  man¬ 
aged  by  AMC  for  the  purpose  of  expanding  its  scope 
to  all  AMC  commands. 

A  principal  objective  of  the  project  was  to  create  a  met¬ 
rics  framework  and  a  prototype  analytical  calculation  and 
aggregation  engine  to  drive  a  near-real-time  tool  (Dashboard) 
for  integrated,  supply  chain  performance  visualization,  with 
supportive  data  reporting.  The  development  of  an  operational  metrics 
system  embedded  in  a  software  program  was  specifically  not  part  of  the 
task.  However,  the  development  of  a  performance  measurement  frame¬ 
work  that  demonstrates  “thought  leadership”  and  advances  in  supply  chain 
performance  metrics  necessitated  that  we  create  a  prototype  system  to 
demonstrate  the  integration  and  comprehensive  features  of  such  a  metrics 
framework.  This  project  was  to  look  beyond  and  add  to  the  metrics  frame¬ 
work  already  adopted  by  DoD. 


DoD  has  previously  adopted  the  supply  chain  operations  reference  (SCOR) 
model  (Supply  Chain  Council,  2016),  but  AMC  wanted  a  fresh  and  inde¬ 
pendent  assessment  of  a  metrics  framework  to  aid  in  redefining  existing 
metrics,  provide  increased  visibility,  and  enable  proactive  action  in  both 
supply  chain  and  acquisition  processes.  Our  team  was  free  to  use  guidelines 
from  SCOR  and  other  metrics  frameworks,  but  we  were  not  limited  to  those 
or  necessarily  required  to  incorporate  those  views  in  this  research  project. 
The  research  team  was  focused  on  developing  a  new  approach  to  integrated 
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supply  chain  metrics.  In  particular,  we  were  specifically  informed 
that  our  task  was  “thought  leadership”  on  the  subject,  and  not 
to  write  executable  code  for  direct  implementation. 


s  the  three-phase  project  progressed,  it  focused  on 
three  dimensions  for  metrics:  on-time  performance, 
efficiency,  and  quality.  On-time  performance  mea¬ 
sures  whether  soldiers  in  the  field  received  items 
ordered  within  a  standard  lead  time  for  the  item. 
Efficiency  focuses  on  the  total  cost  to  the  Army 
for  providing  the  item.  Quality  focuses  on  initial 
quality  and  fielded  life— or  reliability— of  the  item, 
as  well  as  related  quality  costs,  according  to  stan¬ 
dards  for  initial  quality  and  field  reliability. 


The  first  phase  of  the  project  involved  identifying 
and  defining  metrics  that  measure  supply  chain 
performance  of  the  entire  Class  IX  Army  supply 
chain.  Class  IX  items  are  repair  parts  and  compo¬ 
nents  for  all  maintainable  equipment,  ranging  from 
small  parts  to  complex  engines  and  transmissions  for 
—  tanks  and  aircraft.  The  second  phase  of  the  project  inte¬ 

grated  the  metrics  into  a  web-based  visualization  tool— the 
“Dashboard.”  In  the  third  phase,  the  project  sought  to  develop  a 
framework  of  a  broad-based  metrics  system  to 


•  measure  and  evaluate  supply  chain  effectiveness; 

•  communicate  metrics  performance  standards; 


•  direct  attention  to  performance  areas  requiring  management 
intervention; 


•  identify  the  root  causes  of  existing  or  potential  performance 
throughout  the  supply  chain;  and  to 

•  more  effectively  manage  total  ownership  cost. 


Project  success  criteria  were  not  spelled  out  in  a  formal  requirements 
document.  The  sponsoring  lieutenant  general  was  seeking  not  a  specific 
prescribed  outcome  but  a  fresh  look  at  devising  a  metrics  framework  of 
highly  integrated  and  near-real-time  metrics.  With  such  a  framework, 
senior  leadership  could  effectively  assess  areas  of  strong  supply  chain  per¬ 
formance  and  areas  that  need  attention  to  improve  results.  A  dashboard 
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and  drill-down  capability  were  seen  as  desirable  characteristics  of  such 
a  system,  but  how  to  derive  the  functionality  and  what  it  would  look  like 
were  left  to  the  collaborative  work  between  the  research  team  and  Army 
personnel  supporting  the  project. 


Measurement  Challenges 

Measurement  systems  should  be  evaluated  as  a  closed-loop  control 
scheme,  where  multi-organizational-level  and  multi-echelon  metrics  con¬ 
stitute  an  integrated  system  with  performance  specifications  and  feedback 
loops  for  ongoing  and  dynamic  performance  measurement  (Bititci,  Carrie, 
&  McDevitt,  1997;  Melnyk,  Stewart,  &  Swink,  2004).  The  metrics  need  to 
be  aligned  with  internal  organizational  processes  and  external  customer 
and  supplier  performance  using  common  definitions,  data  sources,  and  cal¬ 
culations,  while  balancing  the  tension  between  conflicting  organizational 
priorities  (Melnyk  et  al.,  2005).  Many  organizations  that  seek  to  improve 
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their  performance  measurement  system  have  initially  evolved  to  func¬ 
tion-specific  and  isolated,  or  stovepipe,  metrics,  but  now  want  to  implement 
a  more  integrated  metrics  system. 

For  instance,  Figure  la  (upper  left  corner)  displays  the  existing  metrics 
paradigm  when  the  eSCALE  project  was  initiated  in  2008.  The  metrics, 
for  the  most  part,  existed  in  a  stovepipe  environment,  and  their  definition, 
source  data,  and  interpretation  varied  within  different  AMC  subordinate 
commands.  Metrics  were  collected  and  reported,  and  did  not  reflect  the 
performance  of  the  entire  supply  chain.  Moreover,  the  use  of  averages  often 
hid  issues  that  affect  performance  (Oliver,  Delbridge,  &  Lowe,  1996) 

This  presented  an  opportunity  to  develop  a  standardized  method  of  pulling  and 
analyzing  data  across  the  command  from  multiple  data  systems  to  provide  a 
true  enterprise-wide  view  of  the  AMC’s  supply  chain,  even  when  the  existing 
data  systems  were  unable  to  communicate  across  information  systems. 

The  new  approach  presented  in  the  eSCALE  project,  as  illustrated  in  Figure 
lb,  lc,  and  Id,  involves  moving  away  from  averaging  performance  across  the 
subordinate  commands  to  measuring  and  monitoring  exceptions  in  each 
area  of  the  supply  chain.  This  approach,  combined  with  a  near-real-time 
data  engine,  provides  the  ability  to  identify  and  address  issues  at  the  tac¬ 
tical  level  that,  for  example,  would  show  overstock  or  understock  levels  by 
location,  or  production  lead  times  beyond  acceptable  targets.  Additionally,  it 
provides  the  capability  to  drill  down  to  the  individual  item  level  to  identify 
and  implement  operational  performance  improvements.  This  approach 
identified  issues  with  particular  National  Item  Identification  Numbers 
(NUN)  that  could  be  readily  targeted  for  performance  improvement,  such 
as  the  percentage  of  NIINs  with  administrative  lead  time  (ALT)  or  produc¬ 
tion  lead  time  (PLT)  that  were  beyond  acceptable  standards,  or  stocking 
locations  with  inventory  levels  above  requirements. 

One  of  the  key  contributions  beyond  the  SCOR  approach  was  the  selection 
of  Customer  Wait  Time  (CWT)  for  the  warfighter  (Retail  CWT)  as  a  focal 
metric  for  on-time  performance.  This  is  a  departure  from  more  traditional 
supply  chain  performance  metrics,  such  as  parts  availability  or  inventory 
turns,  and  is  similar  to  a  supply-chain-spanning  version  of  order-to- deliv¬ 
ery  (ODT)  time,  where  we  also  apply  CWT  at  each  upstream  echelon  across 
four  or  more  echelons  in  the  supply  chain  (Keebler,  Manrodt,  Durtsche,  & 
Ledyard,  1999).  Retail  CWT  serves  as  an  objective  surrogate  for  readiness  by 
assessing  on-time  performance  of  supply  delivery  to  warfighters  regardless 


Defense ARJ,  January  2017,  Vol.  24  No.  1 : 66-100 


71 


Increasing  Army  Supply  Chain  Performance 


http://www.dau.mil 


of  their  location  in  the  world.  C  WT  for  upstream  echelons  includes  real-time 
indicators,  before  delivery  of  the  end  item  to  the  warfighter,  on  whether  prior 
processes  are  performing  on  time. 

To  support  the  strategic-level  metric  of  CWT  for  each  echelon  of  the  supply 
chain,  a  suite  of  metrics  was  developed  for  pipeline,  supply,  and  demand 
management  with  tools  for  detailed  analyses  (Figure  lb).  This  approach 
provided  leading  indicators  of  potential  issues  that  would,  if  not  addressed, 
extend  Retail  CWT  for  the  warfighter  beyond  targeted  performance.  These 
metrics  include  measures  of  key  processes  earlier  in  the  upstream  supply 
chain  that  would  give  supply  chain  managers  early  warning  signals  that 
they  need  to  take  some  action  before  they  fail  to  deliver  on  time  to  the 
warfighter.  For  instance,  supplier  delivery  performance  measures  are 
developed  to  serve  as  leading  indicators  of  Supplier  CWT  to  the  wholesale 
supply  chain  that  eventually  affects  Retail  CWT  for  the  warfighter.  In  other 
words,  performance  measurement  of  upstream  echelons  such  as  suppliers 
and  wholesale  distribution  helps  predict  expected  Retail  CWT  for  a  future 
time.  When  the  leading  indicator  falls  out  of  the  range  of  tolerance,  it  will 
be  flagged  to  signal  the  need  for  some  action. 

This  approach  improves  the  current  process,  where  AMC  compiles  metrics 
that  identify  the  number  of  days  the  soldier  waits  for  the  item,  without  the 
ability  to  drill  down  for  issue  investigation.  The  new  approach  also  allows 
monitoring  of  drivers  of  CWT  at  each  upstream  echelon  that  could  extend 
Retail  CWT,  such  as  demand  variation,  increased  lead  time,  bad  data,  poor 
stock  positioning,  delivery  delays,  or  other  root  causes  of  increased  CWT 
(Figure  Id).  Moreover,  existing  metrics  do  not  allow  for  both  a  comprehen¬ 
sive  view  within  a  command  and  an  aggregated  view  across  commands 
along  different  dimensions,  such  as  by  weapon  system  or  Army  Class  of 
Supply  for  parts. 

Another  modification  of  the  SCOR  framework  dealt  with  the  reliability 
metric.  Reliability  in  the  SCOR  model  is  defined  as  “the  predictability  of 
the  outcome  of  a  process”  (Supply  Chain  Council,  2016,  p.  2.1.1),  but  does 
not  apply  to  actual  quality  or  reliability  of  the  product  or  system  deliv¬ 
ered,  as  SCOR  loosely  refers  to  a  product  in  “perfect  condition”  (Supply 
Chain  Council,  2016,  p.  2.1.1).  In  SCOR,  reliability  includes  Perfect  Order 
Fulfillment,  comprising  five  standards  that  include  perfect  quality— which 
primarily  focuses  on  perfect  order  documentation  and  the  product’s  arrival 
in  “good”  condition.  Yet  in  complex  products  and  systems  such  as  those 
deployed  by  the  Army,  reliable  transactional  processes  and  accurate  paper¬ 
work  are  inadequate  to  ensure  product  reliability.  In  the  eSCALE  project, 
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reliability  was  defined  by  the  standard  adopted  by  DoD,  which  specifies  that 
reliability  is  the  failure-free  performance  of  a  system  or  product  for  a  spec¬ 
ified  time  under  specified  conditions  (DoD,  2009;  Information  Technology 
Association  of  America,  2008).  Reliability  in  this  project  includes  initial 
quality  and  fielded  reliability,  using  the  definition  above.  A  detailed  review 
of  the  quality  and  reliability  aspects  of  the  eSCALE  is  beyond  the  scope  of 
this  article  and  is  detailed  elsewhere  (Burns  &  Nicholls,  2013). 

The  objective  ofthe  eSCALE  project  was  to  provide  visibility  across  the  sup¬ 
ply  chain  with  performance  measures  that  afford  an  integrated,  consistent, 
and  near-real-time  view  of  strategic,  tactical,  and  operational  measures. 
With  the  right  measures  and  analytical  capabilities,  individual  supply  chain 
managers  will  be  empowered  with  the  information  necessaryto  identify  and 
resolve  issues,  and  make  informed  resource  allocation  decisions.  Improved 
visibility,  for  example,  supports  improvements  in  inventory  turns,  inventory 
investments,  and  customer  fill  rates  to  soldiers.  Aligning  appropriate  infor¬ 
mation  with  the  existing  organizational  structure  is  expected  to  increase 
visibility  across  organizational  boundaries.  This  will  help  improve  cooper¬ 
ative  and  collaborative  efforts  to  enhance  the  supply  chain  since  increased 
transparency  of  information  becomes  an  instrument  of  change  rather  than 
a  source  of  conflict  within  the  supply  chain. 


FIGURE  1.  INTEGRATED  METRICS  STRATEGY 
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Integration  with  Acquisition 

The  acquisition  process  is  integral  to  the  new  integrated  supply  chain 
performance  measurement  system.  The  new  system  establishes  tolerances 
and  performance  norms  for  the  suppliers,  reports  actual  performance,  and 
highlights  exceptions  to  acceptable  performance.  As  the  system  monitors 
the  supplier’s  performance  on  established  metrics,  AMC  will  have  the 
capability  to  create  a  scorecard  for  each  organic  and  industrial  supplier. 
The  scorecards  can  aggregate  performance  by  Life  Cycle  Management 
Command,  weapon  system,  supplier,  or  from  other  perspectives  for  senior 
leadership  review,  and  they  provide  drill-down  capability  to  the  individual 
item  and  contract  level,  as  well  as  the  aggregated  contractor  report  card. 


It  has  been  known  for  many  years  that  integrating 
sourcing  performance,  such  as  via  acquisition  price 
analysis,  can  have  a  considerable  ejfect  on  overall 
supply  chain  performance  (Beamon,  1998). 

On-time  delivery  performance  may  suffer  due  to  forecasting  error  or  unex¬ 
pected  demand  changes,  which  are  highlighted  in  the  metrics  system.  But 
other  issues  with  poor  CWT  performance  may  arise.  The  drill-down  capa¬ 
bility  of  the  system  allows  the  item  manager  to  identify  issues  that  lead  to 
CWT  violations.  Where  late  deliveries  are  not  caused  by  forecasting  error 
or  unexpected  demand  changes,  the  metric  system  highlights  in  near-real- 
time  other  causal  issues  so  they  can  either  be  corrected  by  the  supplier  or 
contract  modifications  can  be  implemented  before  the  poor  CWT  perfor¬ 
mance  reaches  the  warfighter. 

It  has  been  known  for  many  years  that  integrating  sourcing  performance, 
such  as  via  acquisition  price  analysis,  can  have  a  considerable  effect  on 
overall  supply  chain  performance  (Beamon,  1998).  Excessive  CWT  can 
reduce  readiness  and  increase  costs  by  causing  a  need  for  higher  safety 
stock  inventory.  Quality  and  fielded  reliability  issues  also  increase  cost  by 
requiring  more  items  to  be  provided  to  achieve  an  operational  tempo,  while 
also  increasing  maintenance  and  transportation  costs.  Each  of  these  cost 
effects  can  be  included  in  performance  and  pricing  analysis  at  acquisition, 
and  for  some  suppliers  both  may  be  present  at  the  same  time.  For  acquisition 
evaluation,  the  eSCALE  Dashboard  complements,  but  does  not  incorporate. 
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lagging  performance  data  from  the  Contractor  Performance  Assessment 
Reporting  System  (2016),  Past  Performance  Information  Retrieval  System 
(2016),  or  Governmental-Industry  Data  Exchange  Program  (2016). 

Suppliers  whose  performance  consistently  exceeds  CWT  specifications  can 
be  evaluated  based  on  actual  delivered  costs  that  are  higher  than  quoted 
costs  (Burns  &  Nicholls,  2013),  providing  an  accurate  comparison  of  sup¬ 
plier  performance.  For  specific  supplier  performance,  integrating  these 
costs  across  AMC  commands  rather  than  by  separate  contracts  with  the 
supplier  can  show  aggregated  performance  over  time.  However,  for  full 
implementation,  data  challenges  must  be  addressed  in  the  process. 


Data  Challenges 

There  are  many  Army  data  systems  that  handle  the  complex  transac¬ 
tions  and  processes  necessary  to  support  our  warfighters.  Yet  the  many  Army 
supply  chain  data  systems  are  not  linked  into  a  single  integrated  system  with 
highly  functional  connectivity  to  the  other  systems,  making  comparisons 
of  performance  and  identification  of  improvements  difficult  (Siegl,  2008). 
The  implementation  of  the  Army  Logistics  Modernization  Program  (LMP), 
an  enterprise  resource  planning  system,  for  example,  has  created  a  more 
integrated  planning  system  but  has  not  facilitated  an  integrated,  near-real- 
time,  performance  measurement  system  for  the  Army-wide  supply  chain. 
One  of  the  reasons 
is  that  multiple  data 


systems  are  required 
for  an  integrated  near- 
real-time  system  to  be 
effective,  and  similarly  to 
commercial  systems  that 
have  been  implemented  over 
time  to  serve  different  pur¬ 
poses,  many  of  the  Army  data 
systems  were  not  designed  to 
work  together.  For  example,  in 
an  integrated  system  the  metrics 
should  be  designed  to  work  together 
at  the  strategic,  tactical,  and  opera¬ 
tional  levels,  and  should  use  common 
terminology  and  calculations  for  performance 
measurement  across  all  subordinate  commands. 
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In  the  development  of  the  eSCALE  performance  metric  system,  the  research 
team  integrated  data  inputs  from  more  than  10  different,  nonintegrated  data 
sources  into  a  single  performance  metric  architecture.  The  data  sources, 
records,  and  fields  used  for  each  of  the  calculations  were  developed  by  the 
research  team  and  approved  by  the  Army  sponsor  with  concurrence  from 
Army  subject  matter  experts  (SMEs).  This  avoided  incomplete  data  sets 
from  one  source,  while  providing  a  single  Army-authorized  data  source  for 
each  of  the  calculations. 

Delivered  Results 

By  the  end  of  the  project,  the  architecture  and  framework  for  a  full, 
on-time,  performance  metrics  suite  had  been  developed  and  delivered,  along 
with  a  corresponding  integrated  framework  for  an  efficiency  metrics  suite. 
The  architecture  for  the  quality  and  reliability  framework  had  also  been 
developed,  but  had  not  been  prototyped  and  integrated  with  the  on-time 
performance  and  efficiency  metrics  due  to  funding  reduction  as  a  result  of 
sequestration.  More  discussion  on  these  results  and  the  subsequent  devel¬ 
opment  of  the  quality  and  reliability  metrics  appears  later  in  this  article. 
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Literature  Review 

The  scale  and  complexity  of  Army  supply  chains  necessitate  a  large  and 
complex  organization  to  oversee  them.  Over  decades,  organizational  units 
with  responsibility  for  various  activities  have  been  created,  merged,  and 
in  some  cases  dissolved,  leading  to  a  supply  chain  governed  by  historical 
evolution  and  layers  of  regulation,  policy,  and  directives.  Each  subordinate 
command  specializes  in  supply  chain  management  activities  specific  to 
particular  weapon  systems,  so  not  surprisingly,  many  of  the  individual  orga¬ 
nizational  units  focus  very  specifically  on  their  own  domain  of  responsibility. 

As  a  result  of  the  need  for  specialization,  over  many  years  this  separation 
of  command  responsibilities  has  resulted  in  a  set  of  practices  that  can  be 
viewed  as  “silos  of  excellence.”  One  might  wonder:  would  a  consolidated 
distribution  agency  such  as  Defense  Logistics  Agency  (DLA)  solve  the  silo 
challenge?  Not  likely,  as  the  solution  is  not  that  simple.  While  DLA  has 
responsibility  for  sourcing  and  distributing  many  national  stock  numbers 
that  flow  through  AMC,  it  also  interacts  with  many  suppliers,  technical 
engineering  activities,  and  customer  groups.  Also,  many  of  the  issues  arise 
from  complex  and  long-lead-time  systems  such  as  engines  and  transmis¬ 
sions,  which  require  significant  technical  support  over  time.  From  this, 
our  conclusion  would  be  that  the  metrics  framework  may  be  of  consider¬ 
able  use  to  agencies  such  as  DLA.  The  multi-echelon  integration  across 
diverse  data  platforms  is  one  of  the  strengths  of  the  metrics  framework  and 
visual  dashboard.  Further  benefits  of  this  will  be  explained  in  detail  in  the 
Methodology  section. 

Due  to  the  silos  of  functional  focus  that  have  evolved  over  many  years,  it  is 
often  difficult  for  management  to  obtain  a  consistent  view  or  a  comprehen¬ 
sive  assessment  of  overall  supply  chain  performance  due  to  variation  in  data 
sources,  definitions,  and/or  interpretation.  There  are  many  performance 
measurement  models  in  use,  some  widely  used  and  others  that  are  organiza¬ 
tion-specific  (Bititci,  Carrie,  &  Turner,  2008).  One  widely  adopted  model  is 
the  SCOR  model.  The  SCOR  model  helps  organizations  to  focus  performance 
measurement  on  five  key  processes  across  supply  chains:  plan,  source,  make, 
deliver,  and  return  (Siegl,  2008).  Another  popular  model  is  the  balanced 
scorecard  developed  by  Kaplan  and  Norton  (Kaplan  &  Norton,  1992),  which 
links  performance  measures  from  the  financial,  customer,  internal  business 
process,  and  innovation  and  learning  perspectives. 
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Integrating  and  updating  key  business  processes,  avoiding  inefficiencies 
and  resource  conflict  across  the  supply  chain,  can  be  even  more  challenging 
when  diverse  organizations  develop  their  own  unique  objectives  and  corre¬ 
sponding  metrics  (Henderson,  1994).  This  required  a  strategic  rethinking 
of  the  entire  approach  to  AMC  enterprise-wide  performance  metrics  as  an 
integrated  system.  One  way  to  view  this  approach  was  explained  by  Francis 
(2001),  who  argued  for  increasing  operational  agility  to  improve  perfor¬ 
mance,  using  the  example  of  lessons  learned  from  military  agility  in  the 
field.  While  extended  lead  times  and  other  physical  supply  chain  constraints 
make  it  challenging  to  achieve  agility  with  all  Class  IX  parts,  lessons  can 
be  applied  for  those  items  that  specifically  require  higher  degrees  of  agility. 

The  development  of  an  integrated  suite  of  supply  chain  metrics  should 
include  strategic  measures  of  on-time  performance,  quality  performance, 
and  cost  performance.  A  metrics  suite  should  also  include  drill-down 
subordinate  metrics  to  support  tactical-  and  operational-level  decisions 
(Gunasekaran,  Patel,  &  Ronald,  2004).  This  integrated  cross-organizational 
approach  supports  the  Joint  Supply  Chain  Architecture  (JSCA)  initiative 
(Fletcher,  2011),  which  has  adopted  the  CWT  metric  to  measure  speed  of 
delivery  to  warfighters  (DoD,  2012,  Chapter  15.4). 


Methodology 

The  methodology  for  proposing  a  solution  was  selected  from  a  variety 
of  options.  The  first  step  for  the  Army  was  to  choose  a  partner  with  sig¬ 
nificant  supply  chain  expertise  and  knowledge  of  Army’s  supply  chains, 
but  also  one  that  has  the  ability  to  bring  diverse,  unbiased  perspectives 
and  solutions  to  existing  requirements.  A  UAH  team  of  academic  faculty, 
research  scientists,  and  graduate  students  from  fields  such  as  operations 
research,  operations  management,  supply  chain  management,  marketing, 
economics,  information  systems,  systems  engineering,  reliability,  and 
modeling  and  simulation  was  selected  to  support  the  project.  To  shorten 
the  learning  curve,  AMC  scheduled  joint-learning  seminars  with  the  UAH 
research  team  and  AMC  supply  chain  managers,  supply  chain  specialists, 
and  representatives  from  subordinate  commands  and  organizations,  such  as 
Integrated  Material  Management  Center,  Security  Assistance  Management 
Directorate,  Logistics  Support  Activity,  Rock  Island  Arsenal,  Corpus  Christi 
Army  Depot,  and  AMC  G6  and  G4  to  discuss  the  overall  approach  and 
unique  aspects  of  the  Army’s  supply  chain.  SCOR  principles  were  quickly 
determined  to  be  a  key  element  in  view  of  DoD’s  adoption  of  the  framework. 
It  was  specified  that  the  solution  would  need  to  utilize  existing  data  sources 
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rather  than  create  requirements  for  new  data  sources.  This  task  was  com¬ 
plicated  by  the  necessity  to  accommodate  AMC’s  recent  transition  to  a  new 
SAP-based  information  system  known  as  LMP  (Logistics  Modernization 
Program),  which  at  the  beginning  of  this  multiyear  project  was  a  new 
deployment  at  Aviation  and  Missile  Command  (AMCOM).  To  aid  in  this 
task,  potential  solutions  were  discussed  and  analyzed  during  frequent 
brainstorming  sessions  between  UAH  and  government  representatives. 

The  UAH  team  understood  the  significant  differences  in  the  Army  supply 
chains,  which  are  geared  towards  rapid  deployment,  sustainment,  and 
drawdown  for  wartime  activities.  This  is  different  from  commercial  organi¬ 
zations,  which  tend  to  focus  on  market-based  metrics  such  as  profits,  return 
on  investment  (ROI),  and  stock  prices,  and  often  experience  considerably 
smaller  variations  over  time  in  the  scale  of  operations.  To  bridge  this  dif¬ 
ference,  UAH  and  AMC  officials  conducted  phone  interviews  and  site  visits 
with  several  multinational  commercial  organizations  known  to  apply  best 
practice  supply  chain  strategies.  The  initial  seminars,  meetings,  and  site 
visits  resulted  in  an  agreed-upon  solution  framework,  which  is  represented 
in  Figure  2.  This  article  focuses  primarily  on  CWT  as  an  example  and 
primary  metric  for  on-time  performance,  but  all  of  the  metrics  in  Figure 
2— and  more— were  included  in  the  eSCALE  performance  metrics  system. 
Space  limitations  prohibit  a  full  review  of  all  of  the  elements  of  the  metrics 
project,  which  when  documented  filled  a  book. 


FIGURE  2.  PERSPECTIVES  OF  THE  eSCALE  MEASUREMENT 
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Solution  Perspective  and  Framework 

A  major  objective  in  developing  the  end-to-end  metrics  suite  was  to 
adopt  a  comprehensive  and  multidimensional  approach  to  measure  and 
evaluate  overall  supply  chain  performance  along  the  dimensions  of  on-time 
performance,  efficiency,  and  quality.  On-time  performance  of  the  supply 
chain  is  driven  by  the  chain’s  ability  to  deliver  to  the  end  customer  on  a  time 
standard,  specifically  focusing  on  speed.  Based  on  this  notion,  the  approach 
in  developing  on-time  performance  metrics  places  CWT  as  a  key  metric, 
with  the  rest  of  the  suite  consisting  of  metrics  that  are  likely  to  affect  the 
key  metric  of  CWT.  Accordingly,  the  on-time  performance  metrics  suite 
provides  a  framework  for  insightful  analysis  of  how  on-time  performance 
is  impacted  by  key  drivers  such  as  stock  positioning,  transportation,  timely 
action  by  administrators  and  suppliers,  data  quality,  and  supplier  delivery 
performance.  In  other  words,  CWT  can  be  affected  by  factors  such  as  where 
the  stock  is  positioned  in  the  supply  chain,  the  speed  with  which  the  item 
was  shipped  to  other  members  of  the  supply  chain,  and  the  timeliness  of  the 
procurement  decisions  made  by  the  item  manager.  To  assess  the  impact  of 
on-time  performance  drivers,  we  measure  and  report  metrics  such  as  req¬ 
uisition  wait  time  (RWT),  demand  variation,  stock  position,  and  delivery 
schedules,  as  detailed  in  the  next  section. 


The  Army’s  increased  attention  to  inventory  levels  has  elevated  the  impor¬ 
tance  of  the  efficiency  metric.  A  recent  Government  Accountability  Office 
study  (Solis,  2009)  reported  an  annual  average  of  about  $16.3  billion  in 
secondary  inventory,  of  which  $3.6  billion  (22%)  exceeded  current  require¬ 
ments.  The  study  also  revealed  that,  while  the  Army  experienced  excess 
secondary  inventory,  the  Army  had  annual  inventory  deficits  based  on 
requirements  of  about  $3.5  billion  per  year.  The  lack  of  alignment  of  existing 
inventory  and  current  requirements  was  attributed  to  a  lack  of  cost-effi¬ 
ciency  metrics  and  goals,  and  inaccurate  demand  forecasting. 

Our  research  revealed  that  inventory  management,  management  costs, 
operations  costs,  and  total  ownership  costs  (General  Accounting  Office, 
2003)  are  the  primary  drivers  of  efficiency  in  the  Army  supply  chain.  Metrics 
such  as  inventory  excess,  inventory  allocation,  and  cost  tradeoffs  (e.g.,  repair 
and  procurement)  were  developed  and  measured  to  assess  the  status  of  the 
drivers  of  supply  chain  efficiency. 

Metrics  for  measuring  quality  seek  to  determine  whether  assets  are  func¬ 
tioning  and  performing  in  the  field  as  designed  and  manufactured.  The  key 
drivers  are  initial  quality,  reliability,  and  performance  of  existing  processes. 
Initially,  the  analysis  focuses  on  supply  deficiency  reports  and  quality 
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deficiency  reports.  Additionally,  first  article  test  failures  and  waivers  can 
serve  as  metrics  to  assess  the  key  drivers.  It  was  necessary  in  eSCALE, 
and  in  a  follow-on  research  project,  to  develop  algorithms  to  measure  total 
costs  for  fielded  reliability  of  secondary  components  based  on  planned  ver¬ 
sus  actual  life,  which  is  important  to  containing  life-cycle  costs  (Burns  & 
Nicholls,  2013). 


Once  the  initial  key  metrics  have  been  formulated,  the  focus  shifts  to 
designing  a  performance  measurement  system  that  will  transition  away 
from  averaging  across  the  entire  inventory  to  defining  acceptable  levels 
of  performance  and  subsequently  counting  the  number  of  “bad  situations” 
(i.e.,  flagged  issues)  for  each  individual  part  that  does  not  meet  the  defined 
standards  for  on-time  performance,  efficiency,  or  quality.  This  strategy  is 
common  across  highly  successful  commercial  organizations. 


The  advantage  of  this  approach  is  that  each  individual  transaction,  such  as 
a  part  requisition,  is  evaluated  against  the  standard  prior  to  aggregating  the 
counts  to  a  metric.  The  counts  can  be  aggregated  from  multiple  perspectives, 
such  as  by  supply  support  activities  (SSA),  weapon  system,  region,  priority, 
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and  supplier,  as  well  as  by  key  locations  in  the  supply  chain,  to  show  where 
the  greatest  concentration  of  supply  chain  improvement  opportunities 
exists.  The  next  section  will  detail  the  specific  approaches  used  to  develop 
the  on-time  performance  component  of  the  project. 

The  architecture  of  the  metrics  framework  was  designed  for  multiple  stake¬ 
holders.  For  item  managers,  the  framework  identifies  performance  metrics 
and  issues  that  will  help  them  manage  and  execute  their  tasks  for  NIINs 
for  which  they  are  responsible.  This  is  provided  in  the  drill-down  capabil¬ 
ity  to  the  item  level.  For  senior  leadership  and  for  mid-level  managers,  the 
roll-up  capability  aggregates  and  summarizes  performance  by  responsible 
branches,  divisions  and  commands,  as  well  as  by  weapon  system,  organic 
or  industrial  supplier,  and  other  categories  of  review.  For  issues  that  merit 
additional  attention,  the  drill-down  capability  can  identify  areas  of  strong 
performance  and  areas  needing  additional  attention  at  multiple  levels  of 
scrutiny,  at  the  leader  or  manager’s  discretion. 

During  the  development  and  prototyping  of  the  metrics,  we  had  extensive 
involvement  with  item  managers  and  mid-level  users,  ongoing  periodic 
reviews  with  senior  leadership  to  obtain  feedback,  and  confirmation  of 
definitions  and  calculations  for  metrics  and  for  the  design  of  a  dashboard 
metrics  system  for  their  use.  An  example  of  this  process  is  detailed  in  the 
next  section. 
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One  challenge  that  we  addressed  in  the  Dashboard  design  is  the  issue  of 
trade-offs  of  on-time  performance,  efficiency,  and  quality.  A  well-designed 
metrics  system  identifies  and  highlights  such  issues  for  decision  makers. 
One  advantage  in  the  Army  is  that  the  hierarchical  command  structure 
naturally  allows  for  issues  regarding  performance  trade-offs  to  be  identified 
to  leadership,  where  well-informed  leaders  can  make  decisions  on  the  trade¬ 
offs  that  cannot  be  encoded  into  any  particular  metrics  system.  We  saw  our 
role  as  one  of  helping  to  inform  users,  managers,  and  senior  leadership  of 
areas  of  high  and  of  acceptable  performance,  while  also  highlighting  areas 
where  additional  attention  can  lead  to  increased  performance. 

On-Time  Performance 

Recall  that  the  objective  of  this  research  was  to  develop  a  supply  chain 
system  that  maximizes  overall  performance  of  the  supply  chain  and  the 
related  acquisition  processes.  This  study  proposed  that  on-time  perfor¬ 
mance,  efficiency,  and  quality  are  key  metrics  for  assessing  the  overall 
performance  of  the  Army  supply  chain.  The  focus  of  this  section  is  on-time 
performance,  which  is  driven  by  the  ability  of  the  supply  chain  to  deliver  to 
the  end  customer  on  a  time  standard.  This  details  the  major  steps  adopted 
in  the  development  process  of  the  eSCALE  system  that  measure,  report, 
and  analyze  on-time  performance  of  the  supply  chain.  They  include  the 
following: 

•  Task— Create  a  new  effective  way  to  measure  enterprise  supply 
chain  performance. 

•  Concept— Unmask  and  trace  issues,  and  predict  trends. 

•  Development  of  Logic— Define  tolerances  to  create  standard¬ 
ized  metrics. 

•  Data— Identify  data  sources,  and  acquire  and  verify  data. 

•  Proof  of  Concept— Test  logic  against  the  inventory  to  get  actual 
counts. 

•  Visualization— Create  a  visualization  tool  (Dashboard)  to 
display  results. 

•  Validation  and  Documentation— Assure  data  and  process  con¬ 
sistency  and  reliability  across  commands,  and  prepare  for 
implementation. 
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Task.  The  existing  metrics  program’s  “stovepipe”  environment,  variations 
in  definitions  and  data  availability,  and  dependence  on  the  use  of  averages 
preclude  an  accurate  assessment  of  the  performance  of  the  enterprise-wide 
supply  chain.  Hence,  the  task  for  this  project  was  to  improve  the  existing 
system  by  offering  the  Army  an  integrated  approach  to  measuring  enterprise 
supply  chain  performance. 

Concept.  Given  the  prominence  of  readiness  as  the  definitive  measure  of 
supply  chain  effectiveness,  our  concept  for  addressing  the  on-time  perfor¬ 
mance  metric  focused  on  CWT  as  a  viable  surrogate  for  readiness.  More 
specifically,  we  count  actions,  rather  than  report  averages,  which  poten¬ 
tially  mask  performance  issues.  The  goal  is  to  know  what  percentage  of 
the  NIINs  do  not  meet  established  performance  standards.  Reporting  that 
percentage  is  more  informative  since  the  total  number  can  be  misleading, 
given  the  variation  in  size  of  organizations.  Of  course  we  must  count  both 
total  NIINs  and  NIINs  with  issues  to  determine  the  performance.  Figure 
3  describes  the  counting  logic  process.  The  data  for  NIINs  is  evaluated 
against  Office  of  the  Secretary  of  Defense  (OSD)  standards.  For  each  NIIN, 
if  the  preset  standard  is  being  violated,  a  1  (yes)  is  assigned  and  an  issue 
count  is  generated.  Alternatively,  a  0  (no)  is  assigned  if  the  performance  is 
within  the  standard.  These  results  (0  and  1)  are  rolled  up  at  higher  levels  to 
reflect  tolerance  bands  and  trends,  as  will  be  shown  later  in  the  discussion 
of  the  Dashboard. 


FIGURE  3.  COUNTING  METHODOLOGY:  LOGIC  EXAMPLE 


LOGIC  EXAMPLE 


The  count  standard  assesses  raw  data  against  an  acceptable 
standard.  When  the  data  are  outside  the  standard,  a  bad  ‘count’  is 
generated.  These  are  rolled  up  at  higher  levels  to  reflect 
tolerance  bands  and  trends. 
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Development  of  Logic.  The  next  step  in  the  process  is  the  development  of 
the  logic  underlying  the  system.  This  step  defines  the  tolerances  to  create 
standardized  metrics  across  the  commands.  Standards  were  established 
for  CWT  and  measures  of  major  factors  affecting  CWT,  including  ALT  and 
PLT,  data  discrepancies,  demand  quantity  variation,  delivery  performance 
for  new  buys,  maintenance  and  operations  (M&O)  delivery  performance  for 
both  commercial  and  organic/depot  suppliers,  stock  positioning,  and  RWT. 


Supply  chain  managers  make  decisions  about  item  deliveries  based  on  the 
contract  delivery  schedule  (e.g.,  deliver  items  within  60  days  of  contract 
signing).  Yet,  issues  occur  when  the  LMP  data  do  not  accurately  portray  the 
correct  contract  signing  date.  This  issue  is  captured  in  the  metric  of  ALT, 
which  is  the  time  between  the  date  the  item  manager  requests  the  item  and 
the  date  the  contract  with  the  delivery  schedule  is  signed.  The  tolerance 
metric  for  ALT  states 


•  If  the  deviation  of  average  ALT  is  greater  than  25%  from  the 
LMP  ALT  in  the  last  12  months,  logic  denotes  a  1,  signaling  an 
issue. 
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The  numbers  included  in  the  logic  above  (25%  and  12  months)  and  thereafter 
are  proposed  as  examples  and  should  be  determined  by  proper  decision-mak¬ 
ing  processes  for  the  respective  NIINs. 

Demand  variation  attempts  to  capture  the  extent  to  which  the  current 
demand  is  either  the  highest  or  lowest  value  compared  to  the  previous  12 
months.  Tolerances  might  include  metrics  such  as 

•  If  (the  current)  demand  is  greater  than  monthly  demand  for  each 
of  the  recent  X  months,  the  logic  denotes  a  1,  signaling  an  issue 
(a  change  in  demand). 

•  If  (the  current)  demand  is  less  than  monthly  demand  for  each 
of  the  recent  X  months,  the  logic  denotes  a  1,  signaling  an  issue. 

•  If  demand  is  trending  upward  or  downward  for  Y  consecutive 
months,  the  logic  denotes  a  1,  signaling  an  issue. 

Late  deliveries  are  captured  in  the  delivery  tolerances  for  new  buys,  for  both 
commercial  and  organic  M&O  deliveries.  Examples  of  tolerance  statements 
include 


•  If  the  percentage  of  late  deliveries  for  the  current  month  is 
greater  than  10%,  logic  denotes  a  1,  signaling  an  issue. 

•  If  the  percentage  of  cumulative  late  deliveries  is  greater  than 
10%,  logic  denotes  a  1,  signaling  an  issue. 

Using  the  flags  by  NIIN  (where  a  signal  =  1),  item  managers  can  quickly 
identify  exceptions  requiring  attention.  Inventory  issues  are  related  to  how 
item  managers  administer  their  contract  and  procurement  plans.  The  item 
manager’s  contract  and  procurement  plan  reflects  how  the  item  manager 
plans  to  fulfill  the  requirement  for  a  particular  item  or  component  beyond 
the  scheduled  due-ins.  The  due-ins  plus  the  items  programmed  in  the  con¬ 
tract  and  procurement  plan  should  equal  the  forecasted  requirements.  The 
eSCALE  system  identifies  data  gaps  between  the  demand  forecast  and  the 
total  supply  expected  from  the  due-in  and  the  contract  and  procurement 
plan.  The  tolerance  statement  is 

•  If  the  difference  between  the  total  of  all  due-ins,  contract  and 
procurement  plans,  and  the  forecasted  requirements  is  more 
than  Z  units  in  month  1  of  the  planning  horizon,  the  logic  denotes 
a  1,  signaling  an  issue. 


86 


Defense ARJ,  January  2017,  Vol.  24  No.  1 : 66-100 


January  2017 


Since  serviceable  items  should  always  be  located  as  close  as  possible  to 
the  customer  and  repairable  items  should  be  located  at  the  appropriate 
repair  facility,  a  major  goal  for  the  new  system  is  to  identify  where  inven¬ 
tory  might  be  better  positioned  in  the  pipeline.  For  instance,  a  serviceable 
depot-level  repairable  item  located  in  an  organic  repair  facility  rather  than 
the  SSA  where  it  could  be  issued  to  the  customer  would  be  considered  an 
issue.  Moreover,  an  unserviceable  item  that  can  be  repaired  only  by  the 
contractor  should  be  located  at  the  contractor  repair  facility  rather  than  the 
organic  repair  facility,  the  SSA,  or  an  area  operated  depot  (AOD).  Figure  4 
demonstrates  how  the  eSCALE  system  identifies  stock  locations.  A  flag  of 
1  represents  inventory  that  can  be  better  positioned. 


FIGURE  4.  PERFORMANCE  PERSPECTIVE:  LOGIC  STATEMENTS  || 

Stock  Positioning:  Part  Type 

Identifying  where  inventory  position  can  be  improved  in  the  pipeline 

Contractor 

Repair 

Facility 

Organic 

Repair  SSA 

Facility 

AOD/CCP 

DLR 

Serviceable 

1 

1 

0 

0 

DLR 
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Repair  Only 
Unserviceable 

0 

1 

1 

1 

Contractor 
Repair  Only 
Serviceable 

1 

1 

0 

0 

Note.  DLR  =  Depot  Level  Reparable;  FLR 

=  Field  Level  Reparable. 

A  more  unusual  stock-positioning  challenge  relates  to  stock  positioning 
at  the  SSA.  One  challenge  occurs  when  there  is  unforeseen  demand  for  an 
item  not  on  the  Authorized  Stock  List  (ASL).  The  second  challenge  occurs 
when  items  continue  to  exist  on  the  ASL  (Requisition  Objective  >  0)  when 
there  is  no  current  demand  for  the  item.  The  tolerance  statements  for  these 
two  conditions  follow: 

•  If  the  Requirement  Objective  (RO)  =  0  and  demand  is  greater 
than  3  in  the  past  year,  logic  denotes  a  1,  signaling  an  issue. 
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•  If  the  Requirement  Objective  (RO)  is  greater  than  0,  and  demand 
=  Oin  the  past  year,  logic  denotes  a  1,  signaling  an  issue. 

The  amount  of  time  that  the  SSA  waits  to  receive  the  requisitioned  item 
from  the  wholesale  organization,  such  as  AMCOM,  is  RWT.  RWT  is  also 
an  important  factor  in  determining  the  amount  of  time  the  final  customer 
must  wait  for  an  item  or  component,  CWT.  Hence  the  following  tolerance 
was  established: 

•  If  the  requisition  wait  time  is  longer  than  TRANSCOM 
[Transportation  Command]  regulation  for  CONUS/OCONUS 
[Continental  U.S./Outside  Continental  U.S.],  the  logic  denotes 
a  1,  signaling  an  issue. 

CWT  is  the  seminal  variable  in  the  eSCALE  system,  since  it  is  our  surrogate 
for  readiness.  The  variables  used  in  the  model  discussed  above  (e.g.,  demand 
variation,  stock  positioning)  were  selected  because  of  their  potential  neg¬ 
ative  impact  on  CWT.  The  customer  is  the  warfighter,  who  secures  the 
product/item/component  from  the  SSA.  The  ordering  process  begins  when 
the  soldier  completes  the  appropriate  order  paperwork.  The  resulting  req¬ 
uisition  records  the  date  and  time  the  requisition  was  filed  at  the  SSA.  The 
process  is  completed  when  the  soldier  receives  the  item  from  the  SSA.  The 
time  between  the  date  and  time  of  the  requisition  submission  and  the  date 
and  time  the  soldier  received  the  item  is  the  CWT.  While  most  stock  items 
are  available  for  immediate  distribution  to  the  customer,  the  eSCALE  sys¬ 
tem  identifies  instances  when  the  CWT  exceeds  the  established  tolerances. 

•  If  the  Requisition  Objective  (RO)  is  greater  than  0  and  more  than 
15%  of  the  requisitions  for  an  item  took  longer  than  3  days,  logic 
denotes  a  1,  signaling  an  issue. 

•  If  the  Requisition  Objective  (RO)  =  0  and  more  than  15%  of  the 
requisitions  for  an  item  took  longer  than  TRANSCOM  stan¬ 
dards,  logic  denotes  a  1,  signaling  an  issue. 

The  metrics  discussed  above  were  developed  out  of  many  meetings  and  dis¬ 
cussions  with  different  Army  agencies  across  the  supply  chain.  For  metrics 
to  be  useful,  they  must  be  supported  by  data  that  are  accessible,  accurate, 
consistent,  and  current. 
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Data  and  Proof  of  Concept.  The  development  of  metrics  and  the  logic 
for  metrics  calculations  and  flags  for  performance  improvements  cannot 
be  separated  from  the  process  and  effort  of  identifying  data  sources,  and 
acquiring  and  verifying  data.  The  research  team  collaborated  closely  with 
Army  SMEs  in  this  effort,  with  approvals  by  senior  leadership. 

Since  the  Army  supply  chain  consists  of  organizations  that  are  specialized 
in  their  functional  areas,  it  soon  became  evident  that  identifying  data 
sources  for  the  metrics  was  the  first  essential  step.  Working  groups  were 
formed,  with  representatives  who  are  mostly  SMEs  from  organizations 
managing  or  supporting  the  supply  chain,  to  reach  agreement  on  metrics 
and  standards  for  the  supply  chain  Dashboard  and  to  identify  data  that  will 
be  used  to  calculate  metrics  and  data  sources.  Once  the  data  sources  for 
metrics  were  identified,  more  time-consuming  tasks  followed  our  acqui¬ 
sition  of  sample  data  for  analysis  and  testing  of  proposed  metrics  to  prove 
the  concept. 

Proof  of  concept  involved  testing  the  logic  against  the  available  data  and 
evaluating  the  logic  with  Army  SMEs.  The  tasks  were  not  in  sequential 
order— it  took  a  lot  of  back  and  forth  among  the  UAH  team  and  Army  com¬ 
mands.  Iterative  steps  ensued  during  this  collaborative  process,  from 
defining  metrics  to  identifying  data  sources,  acquiring  data,  verifying 
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data,  testing  metrics  with  acquired  sample  data,  and  testing  metrics  with 
full-scale  data.  It  was  essential  to  do  this  in  order  to  obtain  a  consistent, 
enterprise -wide  view  and  a  comprehensive  performance  assessment  of  the 
supply  chain. 


Visualization.  A  new  visualization  tool,  the  Dashboard,  was  developed  to 
facilitate  a  departure  from  reporting  averages  across  the  entire  inventory  and 
focus  on  counting  of  “flagged”  areas  in  the  individual  supply  chain  elements 
with  the  ability  to  drill  down  to  the  individual  item  level.  Figure  5  provides 
an  example  of  the  Dashboard,  which  includes  the  standards  and  logic  for 
the  CWT  metric.  To  support  the  drill-down  and  root-cause  analysis,  the 
Dashboard  presents  the  total  number  of  issues  by  category  for  each  month 
and  year,  and  the  specific  number  of  issues  by  potential  causes,  such  as 
demand  variation  and  stock  positioning.  Possible  sources  of  issues  related  to 
the  various  causes  of  CWT  are  also  listed.  This  top-level  chart  summarizes 
all  issues  across  the  various  categories.  This  particular  chart— showing  only 
notional  data  for  confidentiality  reasons— reports  at  the  bottom  left  a  total 
of  25,070  NIINs  for  the  requested  supplier,  dates,  and  maintenance  codes. 
Of  the  25,070  NIINs,  5,007  have  at  least  one  issue,  such  as  late  delivery  or 
a  documentation  issue.  Hence,  20,063  have  no  (0)  issues.  A  further  reading 
reveals  that  3,322  NIINs  have  one  issue  and  4  NIINs  have  five  issues.  You 
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will  note,  that  in  this  particular  analysis,  most  issues  occur  because  of  data 
issues  (“Bad  Data”),  improvable  stock  positioning  (“Stock  Positioning”),  and 
excessive  time  to  process  requisitions  (RWT). 
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FIGURE  5.  PERFORMANCE  PERSPECTIVE: 
DASHBOARD  DEVELOPMENT 
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Individual  columns  illustrate  specific  metric 
areas.  For  M&O  Organic,  84  NIINs  had  greater 
than  10%  delivery  delays  out  of  3,351. 


Note.  DODAAC  =  Department  of  Defense  Activity  Address  Code,  Spares  =  supply  problem 
analysis,  reporting  and  categorization  system. 


Figure  6  represents  the  Dashboard’s  drill-down  capability,  which  provides  a 
more  detailed  presentation  of  bad  data  issues  related  to  ALT  and  PLT  for  the 
CZ  AX-99  Airframe  (actual  weapon  system  code  disguised).  In  this  notional 
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example,  there  are  91  NIINs  reported.  Seventy-one  have  at  least  one  bad 
data  issue  related  to  ALT  or  PLT,  so  only  20  NIINs  have  zero  (0)  ALT  or  PLT 
issues.  Eighteen  NIINs  report  one  issue  and  53  report  two  issues.  Of  the  18 
NIINs  with  only  one  issue,  13  are  data  discrepancies  related  to  ALT  and  11 
are  data  discrepancies  related  to  PLT. 


Note.  LAC  =  latest  acquisition  cost,  NMCS  =  not  mission  capable  supply. 


To  examine  additional  drill-down  capabilities  to  understand  issues,  using 
an  example,  the  item  manager  simply  clicks  on  the  number  “13”  under  ALT 
in  Figure  6  to  review  details  on  the  13  NIINs  with  ALT  issues.  Figure  7 
presents  analysis  of  the  13  NIINs  where  the  actual  ALT  dates  differ  from 
the  system’s  data  dates  by  more  than  25%,  a  preset  level  of  tolerance.  This 
report  reveals  each  of  the  13  NIINs  with  both  a  distribution  and  numerical 
percentage  deviation  between  the  actual  ALT  and  the  ALT  reported  in  the 
LMP  data  files.  For  instance,  the  percentage  deviations  between  the  actual 
ALT  and  the  ALT  from  the  data  file  are  shown  in  the  highlighted  (and  dis¬ 
guised)  NUN  on  the  right  (83609549146)  to  be  41%.  Since  41%  is  greater 
than  the  acceptable  tolerance  of  25%,  it  is  flagged  on  this  screen  and  the  item 
manager  can  investigate  the  issue  for  that  particular  NUN  by  clicking  on  the 
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NUN  for  additional  drill  down.  The  results  for  Figure  8  show  that  the  NIIN 
is  for  a  spherical  gear  for  weapon  system  CZ.  The  actual  ALT  is  28.8  days 
and  the  LMP  system  ALT  (days)  is  20.4  days  (data  disguised).  The  manager 
should  investigate  the  causes  of  this  discrepancy. 


FIGURE  7.  PERFORMANCE  PERSPECTIVE: 

BAD  DATA  DRILL  DOWN— ALT/PLT:  ALT 

Bad  Data:  ALT/PLT:  ALT 
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Note.  Nomen  =  nomenclature,  SOS  =  source  of  supply. 

Validation  and  Documentation.  Before  the  Dashboard  can  be  imple¬ 
mented,  steps  must  be  taken  to  assure  data  and  process  consistency  and 
reliability  across  commands.  Data  validation  is  a  major  milestone  before  tool 
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deployment.  Validation  means  signing  off  on  the  data— each  data  element- 
being  analyzed  in  the  Dashboard,  including  calculations,  data  sources,  and 
data  analysis.  Validation  can  also  include  the  Dashboard  layout  and  howthe 
data  are  presented.  A  data  validation  plan  should  be  made  to  determine  the 
validation  process,  approach,  and  data  teams. 

Documentation  provides  a  path  to  communicate  our  methodology  and  to 
meet  Army’s  information  assurance  certification  requirements.  It  also  helps 
the  data  validation  process.  We  documented  the  project  with  specifications, 
data  structures,  logic  algorithms,  and  user  applications  and  instructions  to 
help  educate  item  managers  and  other  users  on  the  capabilities  and  inter¬ 
faces  of  the  performance  metrics  system  framework. 


Benefits  and  Application 

The  delivered  architecture  for  howto  implement  the  improved  metrics 
framework  has  allowed  the  integration  of  existing  metrics,  such  as  those 
adopted  through  SCOR,  with  improvements  such  as  a  focus  on  CWT  that 
is  integrated  with  efficiency.  In  addition,  through  continued  development 
of  the  quality  and  reliability  metrics  initiated  in  this  project  and  further 
developed  in  a  follow-on  project  (Burns  &  Nicholls,  2013),  algorithms  for 
assessing  the  cost  of  initial  quality  and  fielded  reliability  have  been  devel¬ 
oped  as  a  complementary  metrics  architecture  for  the  more  fully  developed 
on-time  performance  and  efficiency  metrics  in  the  eSCALE  project. 

The  proposed  metrics  system  will  enable  managers  to  shift  their  focus 
to  measuring  and  evaluating  performance,  communicating  performance 
expectations,  directing  attention  to  areas  requiring  management  interven¬ 
tion,  and  helping  to  identify  root  causes  of  performance  challenges  for  both 
AMC  and  its  organic  and  industrial  suppliers.  More  importantly,  it  will  help 
expose  critical  issues  and  provide  the  basis  for  launching  new  initiatives 
for  improving  the  supply  chain.  For  instance,  following  the  identification  of 
challenges,  opportunities,  and  goals,  a  “tiger  team”  of  SMEs  could  be  formed 
within  various  segments  of  the  supply  chain. 

One  such  example  was  a  recent  eSCALE  project  that  conducted  simulation 
studies  providing  insights  on  how  to  improve  supply  chain  efficiency  while 
decreasing  the  time  it  takes  to  deliver  long-lead-time  items.  It  was  observed 
that  lead  times  for  aviation  and  missile  parts  have  been  increasing,  con¬ 
tributing  to  increased  inventory  pipeline  costs.  To  reduce  lead  times  and 
overall  inventory  pipeline  costs,  new  contracting  strategies  were  needed 
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for  deploying  strategic  Work-in-Progress  (WIP)  inventory  at  different 
locations  in  the  upstream  supply  chain.  Strategically  placed  WIP  has  the 
potential  to  increase  the  supply  chain’s  capability  to  meet  demand  surges 
and  meet  weapon  system  readiness  requirements  while  helping  to  reduce 
overall  inventory.  Another  example  of  the  increased  visibility  was  the 
identification  of  issues  to  improve  supply  chain  performance  in  theater  and 
other  areas.  Soon  after  the  issues  were  identified,  item  managers  were  able 
to  enact  strategies  that  resulted  in  real  supply  performance  improvements 
to  our  warfighters. 


The  proposed  metrics  system  will  enable  managers 
to  shift  their  focus  to  measuring  and  evaluating 
performance,  communicating  performance 
expectations,  directing  attention  to  areas  requiring 
management  intervention,  and  helping  to  identify 
root  causes  of  performance  challenges  for  both  AMC 
and  its  organic  and  industrial  suppliers. 

The  simulation  examined  the  impact  of  shifting  the  burden  of  the  safety 
stock,  holding  costs,  and  pipeline  costs  into  the  supply  chain.  The  research 
showed  that  investing  money  in  strategically  located  companies,  upstream 
in  the  supply  chain,  and  holding  more  than  the  minimum  stock  could  lead 
to  shortened  service  times  with  an  increased  capability  to  meet  demand 
surges  and  weapon  systems  readiness  requirements. 

In  the  process  of  developing  the  new  system,  many  such  complexities  were 
revealed  and  led  to  interesting  research  projects  by  the  project  team.  A 
recently  completed  graduate  thesis,  for  instance,  investigates  the  inventory 
stock-out  issue  of  class  IX  items  (Neidert,  2011).  Items  with  stock-out  issues 
are  identified  and  characterized  by  statistical  analysis.  The  study  proposed 
a  framework  and  tool  for  inventory  management  based  on  factors  such  as  the 
inventory  level  and  trend  of  items.  The  stock  position  of  an  item  is  forecasted 
with  relatively  high  accuracy,  which  would  assist  supply  chain  managers  in 
their  inventory  decisions.  For  instance,  using  this  framework  can  help  to 
evaluate  whether  items  are  likely  to  be  overstocked  or  understocked.  The 
model  has  been  well  received,  and  the  framework  is  being  used  as  the  basis 
for  a  more  in-depth  analysis  of  supply  chain  efficiency. 
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While  the  project  team  made  considerable  technical  progress  on  the  devel¬ 
opment  of  a  more  real-time  and  integrated  metrics  system  to  enhance 
the  current  system,  the  team  did  not  address  organizational  and  cultural 
elements  that  are  typical  in  any  large-scale  organization.  However,  our 
observation  is  that  in  the  time  since  the  project,  AMC  and  subordinate 
commands  have  included  insights  from  this  and  other  projects  to  make 
important  progress  on  improving  their  metrics  systems. 


Conclusion 

Integrating  performance  metrics  in  an  end-to-end  supply  chain  requires 
a  conscientious  effort  to  identify,  define,  and  calculate  performance  metrics 
from  authorized  data  sources  to  provide  a  broad  set  of  lower  level  metrics. 
Integrating  lower  level  metrics  into  a  higher  level  metrics  system  requires 
that  data  sources  from  nonintegrated  systems  be  integrated  into  a  system 
that  automatically  pulls  and  accumulates  detailed  data  into  local  and  aggre¬ 
gated  measures  to  provide  cascading  levels  of  detail  that  support  local  and 
leadership  decision  making.  Data  integrity  and  cleansing  are  an  integral 
part  of  the  process,  as  is  reaching  common  agreement— in  this  case  with 
the  Army— on  appropriate  data  sources,  algorithms  to  calculate  metrics, 
and  the  design  of  a  dashboard  that  supports  leadership  decisions  and  per¬ 
formance  evaluation  at  higher  levels,  with  drill-down  capability  for  lower 
level  decision  making. 

Focusing  on  the  end  customer,  such  as  warfighters,  drove  the  development 
of  measures  such  as  CWT.  To  support  the  best  and  most  capable  national 
defense,  we  must  make  sure  our  warfighters  receive  the  supply  support 
they  need  at  the  time  they  need  it.  Making  sure  this  process  works  well  over 
time  requires  a  performance  measuring  and  reporting  system  that  directly 
supports  high  performance  on  measures  such  as  CWT. 

The  Army  and  UAH  partnership  on  the  eSCALE  project  led  to  learning 
and  improvement  from  both  partners  and  resulted  in  the  development  of 
CWT  as  the  primary  high-level  metric  for  on-time  performance,  which  was 
adopted  by  JSCA  (Fletcher,  2011).  Additional  advances  have  been  made  by 
UAH,  with  Army  support,  on  development  of  an  integrated  metrics  system 
that  includes  quality  and  reliability,  and  how  high  CWT  and  lower  levels 
of  quality  and  reliability  negatively  affect  life-cycle  costs,  as  measured  by 
total  ownership  cost  (Peeler,  2003).  Those  advancements  are  the  subject  of 
a  future  article  (see  Burns  &  Nicholls,  2013). 
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